Pilot testing of an Assessment tool of Otolaryngology Residents Performing Thyroid Surgery by Using the Global Rating of Thyroid Surgical Skills

Background: There is a pressing need to evaluate the performance of surgical resident in the operation theater objectively by using a structured assessment tool. This will eliminate the possibility of bias which might be a concern when the assessment is subjective. An Objective Structured Assessment of Technical Skills (OSTAT) based tool was tested using Global Rating of Thyroid Surgical Skills (GRTSS) in terms of feasibility, reliability and construct validly in evaluating otolaryngology head and neck surgery residents at various postgraduate years. Objectives: A prospective observational study using a global rating evaluation form developed in conjunction with the OSATS to assess the thyroid surgical skills of residents. The study was conducted in King Abdul Aziz University hospital, Riyadh, KSA. A total of 23 thyroid surgeries performed by 7 residents at three different levels were assessed. The participants’ scores were calculated as percentages and were entered using SPSS (Statistical Package for the Social Sciences, version 17.0) for reliability and construct validity measures. Methods: Construct validity was established by comparing senior and junior years (P<0.001, mean difference: 11.375, 95% confidence interval 6.25-16.49) and by comparing same residents in two different levels (P<0.005, mean difference: 18.8, 95% confidence interval 7.45-30.14). Proved reliability with a value of Cronbach alpha of 0.707. Conclusions: This pilot study has shown this tool to be a valid, feasible, and reliable instrument to assess the thyroidectomy surgical skills. Potential applications will be useful in tracking resident development throughout postgraduate training and certifying operative skills.